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The  advent  of  wireless  personal  communication  services  in  recent  years 
has  created  a  number  of  challenging  research  problems  in  the  areas  of  com¬ 
munications,  signal  processing  and  networking.  A  major  challenge  in  dealing 
with  the  wireless  channel  has  to  do  with  its  inherent  unreliability.  This  is 
in  contrast  with  wired  networks  in  which  the  physical  loss  is  very  small,  e.g. 
of  the  order  of  10“^. 

It  is  generally  argued  upon  that  in  practice,  wireless  video  applications 
involve  users  moving  at  relatively  slow  speeds,  rather  than  at  tens  of  miles 
per  hour.  A  direct  consequence  of  slow  moving  users  is  that  the  resulting 
channels  suffer  from  slow  fading  and  shadowing  effects.  If  the  condition  of 
this  slowly  changing  channel  is  estimated,  it  is  conceivable  to  adapt/modify 
the  source  coding,  modulation,  channel  coding,  power  control,  or  any  other 
aspects  of  transmission  scheme  to  the  channel  condition.  In  particular,  it 
is  possible  to  vary  the  source  and  channel  coding  bit  rates  according  to 
the  channel  condition,  in  such  a  way  as  to  minimize  the  distortion  of  the 
received  signals.  Indeed,  several  researchers  have  applied  this  idea  to  speech 
[1,  7]  and  image  [6,  2]  transmission  over  the  wireless  links.  In  addition  to 
adapting  to  channel  conditions,  one  can  protect  different  source  bits  using 
unequal  error  protection  (UEP)  schemes  such  as  Rate  Compatible  Punctured 
Convolutional  (RCPC)  codes  [5,  1,  2,  7,  6].  With  the  exception  of  [1]  which 
deals  with  speech,  the  remaining  papers  mentioned  above  explicitly  require 
the  source  coder  to  adapt  to  the  channel  condition.  As  an  example,  in 
[8]  a  whole  new  codebook  might  have  to  be  designed  and  used  in  order  to 
optimally  match  each  new  channel  condition. 

With  the  recent  introduction  of  highly  scalable  video  compression  schemes 
such  as  [3],  it  is  possible  to  generate  one  compressed  bit  stream,  such  that 
different  subsets  of  it  correspond  to  the  compressed  version  of  the  same 
video  sequence  at  different  rates.  Thus,  if  one  uses  such  a  source  coder  in 
the  wireless  scenario,  there  is  no  need  to  change  the  source  coding  algorithm. 
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or  any  of  its  parameters,  as  the  channel  conditions  change.  This  is  particu¬ 
larly  attractive  in  multicast  situations  in  heterogeneous  networks  where  the 
wireless  link  is  only  a  small  part  of  a  much  larger  network,  and  the  source 
rate  cannot  be  easily  adapted  to  the  individual  receiver  at  the  wireless  node. 

Over  the  past  year,  we  have  developed  a  technique  for  optimum  parti¬ 
tioning  of  source  and  channel  coding  bits,  for  the  scalable  video  compression 
algorithm  described  in  [3].  By  “optimum”,  we  mean  a  partitioning  which 
results  in  minimum  expected  value  of  distortion,  which  we  choose  to  be 
Mean  Squared  Error  (MSE).  We  will  consider  the  case  where  the  channel 
state  information  (CSI)  is  known,  and  the  joint  source/channel  codec  will 
adapt  to  the  channel  and  optimally  transmit  video  for  the  current  channel 
state.  In  section  1,  we  briefly  describe  our  source  coding  algorithm,  section  2 
formulates  our  problem  and  outlines  our  basic  approach,  section  3  contains 
details  of  our  optimization  based  approach  and  section  4  includes  results. 

1  Scalable  Source  Coder 

The  source  coder  we  use  in  this  project  is  the  scalable  coder  described  in 
[3].  This  coder  has  been  shown  to  generate  rates  anywhere  from  tens  of 
kilo  bits  to  few  mega  bits  per  second  with  arbitrarily  fine  granularity.  In 
addition,  its  compression  efficiency  has  been  shown  to  be  comparable  to 
standards  such  as  MPEG-1  [3].  The  fundamental  idea  behind  it  is  to  apply 
three  dimensional  subband  coding  to  the  video  sequence  to  obtain  a  set  of 
spatio  temporal  subbands.  Subsequently,  each  subband  coefficient  is  suc¬ 
cessively  refined  via  layered  quantization  techniques.  Finally,  conditional 
arithmetic  coding  is  applied  to  code  different  quantization  layers.  In  doing 
so,  the  spatial,  temporal,  and  inter-subband  correlations,  as  well  as  corre¬ 
lation  between  quantization  layers  are  taken  into  consideration  to  minimize 
the  bit  rate.  The  problem  of  optimal  source  allocation  between  different 
quantization  layers  of  different  subbands,  in  the  absence  of  channel  errors 
has  been  discussed  in  [3].  In  the  next  section,  we  outline  our  approach  for 
the  case  where  channel  errors  are  not  zero. 

2  Problem  Formulation 

The  main  problem  we  solve  is:  given  a  total  number  of  bits  C,  and  a  given 
binary  symmetric  channel  with  bit  error  probability  Pg,  find  the  best  source 
coding  rate  Rs  and  channel  coding  rate  Rc  such  that  C  =  Rs  +  Pc? 
the  expected  value  of  MSE  is  minimized.  This  is  equivalent  to  finding  the 
optimal  source  to  channel  bit  ratio  P^/P^,  with  P^  +  P^  =  C,  such  that 
the  distortion  is  minimized.  To  find  these  minima  for  various  CSPs,  our 
approach  is  to  construct  distortion  curves  )  and  to  locate  the  minima 

empirically. 

In  constructing  the  above  curves  for  each  specific  value  of  we  must 
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answer  two  questions:  first,  which  quantization  layers  of  which  subband 
should  be  included  in  Second,  to  what  extent  should  each  one  be 

protected?  If  njt  denotes  the  number  of  source  bits  in  subband  k  of  the 
source  coder  described  in  section  1,  and  rui^k  denotes  the  number  of  channel 
bits  used  to  protect  source  bit  i  of  subband  A:,  then  our  optimization  problem 
is  that  of  minimizing  the  expected  value  of  MSE  given  by  [9]: 


E[MSE] 

E[MSEk] 


K 


k  =  l 


+  “  9{mj,k)]fk(nk  +  1) 

j=i 


(1) 


(2) 


subject  to  the  constraints: 

K 

y^^rik  <  R, 

k  =  l 


K  n„ 


E 

k=l  t=l 


(3) 


In  the  above  equations,  MSEk  is  the  mean  square  error  of  subband  k, 
K  is  the  total  number  of  subbands,  g[nii^k)  is  the  bit  error  probability  if 
rui^k  channel  bits  are  used  for  protection  of  bit  i  of  subband  A:,  and  fk{i)  is 
the  resulting  MSE  if  an  error  occurs  at  bit  i  of  subband  A:,  or  equivalently 
the  error  sensitivity  function. 

To  simplify  the  above  problem,  we  can  introduce  a  new  variable  rrik  given 
by: 

n/c 

Y^rrii^k-ruk  (4) 

1=1 

to  replace  the  variables  rrii^k-  The  MSE  of  subband  k  can  now  be  represented 
as  a  function  of  2  variables  rik  and  rrik: 

K 

E[MSE]  =  Y,E[MSEk(nk,mk)]  (5) 

k:^l 

However,  there  is  now  an  additional  problem  of  optimally  distributing  rrik 
channel  bits  among  Uk  source  bits,  or  the  mapping  from  rrik  to  the  set 
{rrii^k}^  This  is  a  discrete  non-linear  optimization  problem  which  can  be 
solved  via  the  well  known  branch  and  bound  technique.  To  speed  up  this 
optimization,  we  can  impose  the  following  pruning  rules  for  the  branch  and 
bound  algorithm:  first,  all  bits  within  the  same  quantization  layer  must 
receive  the  same  level  of  protection;  second,  higher  quantization  layers  never 
receive  more  protection  than  lower  quantization  layers. 

In  the  next  section,  we  focus  on  the  problem  of  optimizing  5  subject  to 
the  constraints  in  3. 
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A 

R, 

Rc 

64.0  kbits/s 

67.3  kbits/s 

70.7  kbits/s 

67.9  kbits/s 

0.095 

72.4  kbits/s 

67.9  kbits/s 

0.085 

72.4  kbits/s 

68.4  kbits/s 

0.075 

72.4  kbits/s 

69.3  kbits/s 

Table  1:  Source  and  Channel  Rates  for  various  A  and 


3  OPTIMIZATION  PROBLEM 

Our  solution  to  the  optimization  is  based  on  a  variation  of  Lagrange  Mul¬ 
tipliers,  similar  to  the  one  developed  in  [4],  with  the  exception  that  we  are 
considering  optimization  of  two  sets  of  variables  instead  of  one.  The  general 
theorem  states  the  solution  to  the  unconstrained  optimization  problem: 

{K  K  K  \ 

MSEk[nh,mk)  +  >  (6) 

kzz\  k-\  k  =  \  J 

is  also  the  solution  to  the  constrained  optimization  problem  described  ear¬ 
lier,  provided  there  exists  A  and  fi  such  that  the  constraints  in  3  are  met 
with  equality.  The  questions  left  to  solve  this  unconstrained  problem  are: 
(a)  for  given  A  and  /z,  how  to  find  the  optimal  solution  to  6;  (b)  how  to  find 
A  and  fi  to  meet  the  constraints  stated  in  3.  We  will  answer  those  next. 

3.1  Solving  the  unconstrained  problem 

We  note  that  to  solve  6  for  given  A  and  we  can  solve  for  each  set  of  vari¬ 
ables  {nk^rrik)  independently.  In  other  words,  we  can  solve  K  independent 
minimization  problems  of  the  form: 

mm{MSEk{nk,mk)  -f  Xrik  +  firrik}  (7) 

To  solve  each  optimization  problem,  we  exploit  the  fact  that  the  number  of 
quantization  layers  for  each  subband  takes  on  finitely  many  discrete  values. 
For  instance,  the  source  video  that  is  coded  to  250  kilo  bits  per  second,  can 
have  a  maximum  of  7  quantization  layers  in  each  subband.  This  implies  that 
for  each  subband,  can  take  on  one  of  7  discrete  values.  Our  approach  to 
solving  the  optimization  problem  in  7  is  exhaustive  search.  Specifically,  we 
step  through  all  values  of  Uk  and  for  each  one,  find  the  optimum  rrik  using 
the  branch  and  bound  technique  discussed  in  section  3. 

3.2  Finding  the  Lagrange  Multipliers 

Note  that  for  each  set  of  (A,/i),  we  can  find  the  corresponding  values  of 
i'^ki'rnk)^  and  hence  (Rs^Rc)  using  the  approach  outlined  in  section  3.1. 
As  such,  we  can  claim  that  R^  and  Rc  are  both  functions  of  both  A  and 
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fi.  To  facilitate  our  goal  of  finding  (A,^)  that  meet  the  constraints  in  3 
with  equality,  we  make  the  simplifying  assumption  that  Rs  (  Rc)  is  only  a 
function  of  A  (  /x).  To  show  numerically  that  this  is  a  valid  assumption, 
we  construct  Table  1  for  CSI  of  Pe  =  0.05,  using  the  approach  described  in 
section  3.1.  We  see  that  as  we  vary  /x  and  keep  A  constant,  Rs  stays  relatively 
constant  while  Rc  changes  dramatically;  the  opposite  is  true  when  we  keep 
fi  constant  and  vary  A.  This  shows  Rs  is  more  sensitive  to  changes  in  A  than 
in  /X,  and  the  opposite  is  true  for  Rc^  and  our  assumption  is  approximately 
valid.  Another  important  observation  one  can  make  from  Table  1  is  that  in 
general,  the  rate  Rs  (  Rc)  is  inversely  proportional  to  A  (  /x).  Putting  these 
together,  we  can  propose  the  following  search  strategy: 

L  Guess  an  initial  value  for  A^  and  fj.^ ,  Using  these  values,  find  corresponding 
R]  and  using  procedures  discussed  in  4.1. 

2.  If  target  R^  is  larger  then  R]^  let  A^  =  aA\  where  0  <  a  <  1.  Else  A^  = 
(1/q)A^  Similar  procedure  for  /i^. 

3.  Construct  a  linear  function  Rs{l/X)  of  variable  1/A  using  previous  two  sets 

of  points,  and  Target  R*  by  estimating  A^  using 

this  linear  function.  Similar  procedure  for  /x^ 

4.  If  R^^  is  within  e,  of  R*  and  is  within  Cc  of  stop;  Otherwise,  goto  3. 

Empirically,  we  found  a  =  0,1,  Cg  =  0.05/?^,  and  6c  =  0.05/?*  results  in  15 
to  20  iterations.  Since  we  terminate  the  search  algorithm  when  it  reaches  an 
approximate  solution,  a  natural  question  is,  “how  far  off  is  the  answer  from 
the  ideal  solution?”.  To  answer  this,  we  can  apply  the  lemma  in  [4]  and 
show  that  for  our  experimental  results  described  in  section  4,  our  algorithm 
terminates  when  it  is  within  5%  MSE  of  the  actual  solution. 

4  RESULTS 

To  test  the  above  algorithm  numerically,  we  combine  the  3D  scalable  video 
codec  and  Rate-Compatible  Punctured  Convolutional  Codes  [5]  to  build  our 
proposed  joint  source/channel  codec.  For  source  coding,  we  use  3  levels  of 
spatial  and  2  levels  of  temporal  subband  decomposition.  We  use  200  frames 
of  the  digitized  video  “raiders  of  the  lost  ark”  to  compute  the  distortion 
functions,  and  apply  our  bit  allocation  strategy  to  search  for  the  optimal 
source  to  channel  coding  ratio  R^/Rc  for  various  CSI  ranging  from  0.001 
to  0.05.  The  total  bit  budget  is  250kbits/s,  We  see  in  Figure  1  that  there 
exists  a  unique  distortion  minimum  for  various  Fe* 

We  observe  that  as  the  error  probability  Pe  increases,  the  total  number  of 
quantization  layers  selected  decreases;  this  is  due  to  the  decrease  of  optimal 
source  to  channel  ratio  as  the  channel  condition  worsens.  From  Fg  =  0  to 
Fg  =  0.001,  the  layers  that  are  dropped  are  mostly  high  frequency  layers; 
this  is  due  to  the  low  error  sensitivities  of  high  frequency  components.  In 
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Distortion  vs.  Sourco  to  Channel  Ratio  (250kbits/s) 


Figure  1:  MSE  vs.  Rs/Rc  for  various  CSI 


poor  channel  condition,  >  0.01,  the  number  of  layers  of  low  frequency 
subbands  is  reduced,  resulting  in  a  more  uniform  distribution  of  quantization 
layers  among  the  subbands.  This  is  because  higher  quantization  layers  are 
useless  unless  all  the  preceding  lower  layers  are  received  error-free.  Therefore 
a  subband  with  too  many  quantization  layers  will  render  the  higher  layers 
frequently  futile  in  poor  channel  condition. 

Although  we  assume  the  knowledge  of  the  channel,  there  are  times  when 
the  estimate  of  the  channel  is  incorrect.  Using  Figure  1,  we  can  easily  de¬ 
termine  the  performance  degradations  in  such  situations.  For  example,  to 
find  the  approximate  performance  of  the  joint  source/channel  codec  assum¬ 
ing  Pe  =  0*01  but  operating  at  Pg  =  0.05,  we  locate  the  point  R^/Rc  for 
Pg  =  0.01  on  the  curve  Do.os{Rs/Rc)* 

If  CSI  is  given  in  the  form  of  a  probability  distribution  function,  then 
our  proposed  approach  can  be  used  to  find  the  optimum  operating  point: 


R:/RI  =  arg 


DpARs/Rc)P(CSI 


(8) 


where  F  is  the  set  of  all  possible  CSI’s.  In  this  situation,  our  joint 
source/channel  coding  approach  has  better  adaptation  potential  to  the  chan¬ 
nel  than  previous  schemes  [2]. 

To  show  that  our  optimization  strategy  is  essential  in  poor  channel  con¬ 
dition,  Pg  =  0.05,  we  compare  its  performance  with  other  codecs  in  Figure 
2.  Curve  a  in  Figure  2,  shows  the  PSNR  of  the  scalable  codec  under  ideal 
noiseless  conditions  for  100  frames.  The  average  PSNR  in  this  case  is  31.8 
dB.  Curve  b  in  Figure  2  shows  the  PSNR  of  our  proposed  optimized  codec 
operating  at  the  optimal  Rs/Rc  =  0.6,  with  unequal  error  protection  as  de¬ 
scribed  in  earlier  sections.  The  average  PSNR  in  this  case  is  about  4  dB 
lower  than  the  ideal  noiseless  case.  Curve  c  in  Figure  2  shows  the  perfor- 
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Distortion  vs.  Source  to  Channel  Coding  Ratio  (250Kbit8/s) 


Figure  2:  PSNR  vs  Frames  for  different  Codecs:  a)  noiseless  channel;  b) 
proposed  codec  with  unequal  error  protection  =  0.6,  Pe  =  .05;  c) 

equal  error  codec  R^/Rc  =  0.6,  Pe  =  .05;  d)  equal  error  codec  Rs/Rc  =  2, 
Pe  =  .05 

mance  of  a  codec  operating  at  the  optimal  Rs/Rc  =  0,6,  but  using  equal 
error  protection.  This  codec  distributes  Rs  source  bits  using  traditional  bit 
allocation  theory  that  assumes  a  noiseless  channel,  then  channel  codes  these 
source  bits  with  Rq  channel  bits  equally.  As  seen,  the  PSNR  is  about  the 
same  as  case  b  for  most  frames,  except  for  occasional  drops  of  25  dB.  These 
drops  are  a  direct  consequence  of  the  fact  that  important  source  bits  not 
being  adequately  protected  and  can  result  in  objectionable  degradations  to 
the  quality  of  the  video.  Finally  curve  d  in  Figure  2  shows  the  performance 
of  the  same  equal  error  protection  codec  as  in  c  but  operating  at  non-optimal 
Rs/Rc  =  2.  As  seen,  the  average  PSNR  of  this  codec  is  about  8  dB.  The 
main  conclusion  to  be  drawn  from  Figure  2  is  that  optimal  source/channel 
bit  distribution  does  make  a  significant  difference  in  both  visual  quality  and 
PSNR. 
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